Skip to main content

Blind Spot Analysis

The Blind Spot Analysis tab helps identify subcohorts within a validation where model performance deviates significantly from the overall results. These blind spots can arise from demographic bias, modality variations, or other metadata-related factors. The tool enables users to detect, explore, and validate such regions using flexible cohort generation and performance comparison.

1. Navigation and Validation Selection​

  • Navigate to Validation Analysis → Blind Spot Analysis
  • Choose a validation run from the list to begin analysis
  • The interface lists all available validations, allowing quick switching between runs

2. Generating Blind Spot Analysis​

Once a validation is selected:

  • Click on Generate Blind Spot Analysis
  • Configure parameters:
    • Min Cohort Size: Minimum number of samples required in a cohort
    • Lower Bound: Threshold below which performance drop is flagged
    • Max Combinations: Limits how many cohort combinations to evaluate
  • Optional: Specify relevant metadata fields (e.g., Age, Gender) to control the scope
  • Click Submit to start the analysis
  • While the analysis runs, a loading status is shown

3. Viewing Analysis Results​

Once complete, the interface displays results under a new Round tab:

  • Each round corresponds to a unique configuration of the blind spot analysis
  • The latest round is always shown at the top
  • You can filter by class, or toggle cohort dimensions like Age, Gender, or Ethnicity

4. Filtering and Interacting with Results​

  • Use the Filter panel to customize which metadata combinations appear
  • When selecting a filter (e.g., only Age), the list updates to show only subcohorts including that field
  • The subcohort table updates dynamically:
    • Rows re-order based on filter priority
    • Non-selected metadata columns are hidden

5. Interpreting Performance Charts​

The tool visualizes performance changes across identified subcohorts:

  • Each subcohort is represented as a bar in a chart
  • X-axis shows sample size
  • Bar height reflects performance drop compared to base validation
  • Subcohort size is proportional to sample count
  • Cohorts below the Min Cohort Size threshold are listed separately

6. Subcohort Validation​

Users can generate targeted validation for selected subcohorts:

  • Select one or more subcohorts
  • Click Generate Subcohorts
  • A new validation is triggered for the selected cohorts
  • Once complete, shortcut icons appear next to the subcohort entry
  • Clicking these icons opens the Validation Metrics page for that cohort, enabling deeper analysis

✅ Tip: Use Blind Spot Analysis to proactively detect performance gaps across demographic or technical subgroups, and create validations to confirm model reliability under varying conditions.